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Through a redefinition of patterns in an Hopfield-like model, we introduce and develop an approach 
to model discrete systems made up of many, interacting components with inner degrees of freedom. 
Our approach clarifies the intrinsic connection between the kind of interactions among components 
and the emergent topology describing the system itself; also, it allows to effectively address the 
statistical mechanics on the resulting networks. Indeed, a wide class of analytically treatable, 
weighted random graphs with a tunable level of correlation can be recovered and controlled. We 
especially focus on the case of imitative couplings among components endowed with similar patterns 
(i.e. attributes), which, as we show, naturally and without any a-priori assumption, gives rise to 
small-world effects. We also solve the thermodynamics (at a replica symmetric level) by extending 
the double stochastic stability technique: free energy, self consistency relations and fluctuation 
analysis for a picture of criticality are obtained. 
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The performance of most complex systems, from the 
cell to the Internet, emerges from the collective activity of 
many inner components; At an abstract level, the latter 
can be reduced to a series of nodes that are connected 
each other by links, envisaging the interaction. Nodes 
and links together form a network [THS]. 

The view offered by network description has led to 
identify classes of (topological) universality [4 , and 
to evidence how experimentally-revealable features, e.g. 
cliquishness, modularity, assortativity or peculiar degree 
distribution, not only underlie a certain degree of corre- 
lation among components and/or links but also crucially 
affect the behavior of the system [T] . This constituted a 
real breakthrough with respect to the previous tendency 
to model complex networks either as regular objects, such 
as square or diamond lattices, or as (purely uncorrelated) 
random networks a la Erdos-Renyi (ER) [B]. 

In the first part of this work we show that, within our 
approach, the kind of interaction (e.g. imitative, repul- 
sive, etc.) among components naturally gives rise to a 
broad class of weighted random graphs, whose topolog- 
ical properties can be properly tuned. Such a connec- 
tion also allows to infer about the plausibility of a given 
modclization: the choice of a particular network must be 
consistent with the kind of interactions governing the sys- 
tem itself and viceversa, ultimately reflecting the internal 
structure of the considered nodes. 

In the second part of this work, we study the thermo- 
dynamic properties of a subset of these structures gen- 
erated by a "Hebbian-like kernel" ; interestingly, such an 
approach allows to work out the thermodynamics of a 
wide class of diluted graphs, even in the presence of fer- 
romagnetic disorder on couplings. 

Modelization. Given a set of V components, each char- 
acterized by a "pattern" £ (similarly to the "hidden vari- 



able" approach [7J |S] ) drawn from a given probability dis- 
tribution, a pair of nodes i and j is linked according to 
a proper rule r(&,£j*). More precisely, here, £ is a vector 
given by binary string of length L, which encodes a set of 
attributes characterizing each node; Then, the function 
r associates each couple of strings to a real value which 
provides the pertaining coupling: r(£j,£j) = Jy. 

Now, crucial for the whole approach are the way the 
strings are generated and the rule r, both to be defined 
according to the processes one wishes to model. 

The Hebbian-like kernel We investigate in detail the 
case of biased patterns where the probability to extract 
any entry is P(£f = 0) = 1 - P(£f = 1) = (1 - o)/2, a g 
[—1,1]; the rule r given by the scalar product among 
strings 

Notice that such a rule resembles the Hebbian kernel 
well-known in neural networks jS], apart from the shift 
[— 1, +1] — ► [0, +1] in the definition of patterns; this plain 
replacement converts frustration into ferromagnetic dilu- 
tion. We are therefore focusing on systems where the 
interaction among components is stronger the larger the 
number of attributes they share. 

An important parameter characterizing a given node, 
is the number p of non-null entries present in the re- 
lated string: Due to the independence underlying the 
extraction of each entry, p follows a binomial distri- 
bution Px(p;a, L) = B(p;L,(l + a)/2) with average 
p a: L = L(l + a)/2. Moreover, it can be shown that 
the probability for two string and £j, displaying re- 
spectively pi and pj non-null entries, to be connected is 
P^{Pi,Pf,L) = l-iL-piYiL-pjy./iLliL-pi-pjy.}. By 
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FIG. 1: (Color on line) Degree distribution Pd eS iee(z; a, L, V) 
for systems displaying small ratio L/V and multimodal dis- 
tribution (left panel) and large L/V with modes collapsing 
into a unimodal distribution. In the former case we plot data 
points (•) as well as the the analytical curve provided by 
Eq. [2] In the latter case we show the distributions corre- 
sponding to each mode Pa egr ee(z; a, p,V) in agreement with 
numerical data, as well as the overall distribution (thicker, 
bright curve). 



averaging over, say pj, one finds the expected link proba- 
bility flinkOi, a) = 1 - [(1 - a)/2]« for the node i. Then, 
the probability that i has degree (number of neighbors) 
equal to z follows as the binomial Pd C grcc{ z , a, pi,V) = 
B{z\ V, Piink(pi, a))- Due to the average over pj, such a 
degree distribution corresponds to a mean-field approach 
where we treat all the remaining nodes in the average; 
this works finely for V large and pi not too small (see 
also Fig. [IJ. Therefore, the number of null-entries con- 
trols the degree distribution of the pertaining node: A 
large p gives rise to narrow (small variance) distributions 
peaked at large values of z. Then, the global degree dis- 
tribution reads off as a combination of binomials 



-Pdcgrcc(z; a, L, V) 



^ Pdc gICC (z; a, p, V)P%(p; a, L), 

p=0 



(2) 

giving rise to a i-modal distribution, where "modes" , 
each corresponding to a different value of p, are solved as 
long as the connectivity and L are not too large in order 
to ensure spread distributions for z and p (see Fig. [T] left 
and right panels, respectively and [H] for more details). 

Multimodal degree distributions constitute an inter- 
esting feature of the model as they allow to naturally 
discriminate between different classes of nodes possibly 
fulfilling different functions. In particular, nodes corre- 
sponding to a large degree are often associated to a rel- 
atively small reactivity and vice versa [HI E] ■ 

A more global characterization can be attained by the 
average link probability, applying to a generic couple of 



nodes, neglecting any information about correlations 

. mi 
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so that the average degree reads as z = pV. Now, for 
L — > 00, p approaches a discontinuous function assuming 
value 1 when a > — 1 and value when a = — 1. To fix 
ideas, let us focus on the the so-called high-storage regime 
where L is linearly divergent with V, i.e. L = aV, then, 
the range of values for a yielding a non-trivial topology 
can be characterized by means of the scaling 
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where 9 > and 7 is a finite parameter [20] . 

Following \J2l, , it is possible to distinguish the following 
regimes (see Fig. [2]) : 



<l/2, p«l, 



= 1/2, p 



1 



V 



= - 7 2 q/4 



Fully connected graph 



7 2 a/4, 



0(V) 



=> Linearly diverging connectivity; Within a mean- 
field description the ER random graph with finite 
probability is recovered. 
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limy^oo z~ Y 
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= limy 
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p - 7 2 aV 1 - 2 V4, 
Extreme dilution regime: 
y z/V = Q HO]. 



• 6 = 1, p-^, z = O(V ) Finite con- 
nectivity regime; Within a mean-field description 
7 2 a/4 = 1 corresponds to a percolation threshold. 

Larger values of 9 determine a disconnected graph with 
vanishing average degree. 

Therefore, while 9 controls the connectivity regime of 
the network, 7 allows a fine tuning. 

Up to now we just focused on topological disorder; as 
for couplings, we can still detect "modes", each charac- 
terized by J p representing the average strength for links 
stemming from a node associated to a string with p 
non-null entries. While J p provides a measure of the 
local "field" seen by a single node, a global descrip- 
tion can be attained by the overall average coupling 
J = J2 P JpP\{p\ a i L), taken over the whole graph. The 

two quantities are related via J p = p/lV~J j!2j and, as we 
will see, despite the self-consistence relation (see Eq.10), 
more sensible to local conditions, is influenced by V~J, the 
critical behavior occurs at /3 C = J -1 consistently with a 
manifestation of a collective, global effect. 

By looking in more detail at the coupling distribution 
holding in the thermodynamic limit and for values of a 
determined by Eq.|4j we find that for 1/2 < 9 < 1, nodes 
are pairwise either non-connected or connected due to 
one single matching among the relevant strings, namely, 



3 



a 



4/Y 2 



ig Connectivity 


1 Extreme § 


Connected 'go 
> 

b 
>- 

L. 
« 

c 

Li 


1 Dilution (j 

! 

U- 



1/2 



e 



connected with each other. This is also confirmed nu- 
merically: Fig. [3] shows that c/c ER > 1 in a wide region 
of the plane (a, a), and, in particular, in the region of 
high dilution. 

Interestingly, this kind of link correlation allows to de- 
tect, within the graph, communities densely and strongly 
linked up, so that the role of weak ties is crucial in bridg- 
ing such communities and maintaing the graph connected 

[Hi eng. 

Thermodynamics When dealing with thermodynami- 
cal properties of these networks, we first paste V vari- 
ables a on the nodes and define the Hamiltonian 
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FIG. 2: (Color on line) Phase diagram representing the topol- 
ogy of the graph as the parameters 0, 7 and a are varied. 
In the region on the left (blue) disorder on couplings is still 
present, while on the right side (yellow) disorder on couplings 
is lost while topological inhomogeneity is still present. In the 
narrow central region (green) a coexistence of both kinds of 
disorder is achieved. 



neglecting higher order corrections, J = with proba- 
bility po ~ exp(a7 2 I/ 1 - 2e /4) - 1 - 7 2 a/(4V" 29 ~ 1 ), while 
J = L^ 1 with probability pi ~ po7 2 a/(4F 29_1 ) ~ 1-po- 
For 6 = 1/2 this still holds when a 7 2 /4 < 1, which 
corresponds to a relatively high dilution regime, other- 
wise some degree of disorder is maintained, being that 
Pk ~ (a7 2 /4) fe /fc!. On the other hand, for < 1/2, while 
topological disorder is lost, disorder on couplings is still 
present. However, for 9 = and 7 = 2, the coupling 
distribution gets peaked at J = 1 and, again, disorder 
on couplings is lost so that a pure Curie- Weiss model is 
recovered (see also [T2"]h 

Before concluding this part, we stress that slight vari- 
ations in the rule provided by Eq. [T] can yield dramatic 
changes in the global layout of the graph, e.g. scale-free 
degree and/or coupling distributions. 

Small- World properties A "small-world" network [13] 
displays, by definition, diameter growing as log N, hence 
comparably with the case of ER random graphs, and high 
cliqueshness indicating a high level of redundancy. 

The cohesiveness of the neighbourhood of a node i is 
usually quantified by the (local) clustering coefficient Cj, 
defined as the fraction of permitted edges between nodes 
adjacent to i that actually exist. The average clustering 
coefficient c = Xa=i c « can be compared with the one 
expect for a comparable (i.e. displaying same average 
degree) ER graph, namely c ER = z/V. 

For the graph generated by Eq. [T] the neighborhood 
Vi of i is made up of all nodes displaying at least one 
non-null entry corresponding to any non-null entries of 
£j. This condition biases the distribution of strings rel- 
evant to nodes € Vi, so that they are more likely to be 



formally identical to an Hopfield model, apart the 
normalization which accounts for not symmetrical 
w.r.t. zero quenched variables. Via Eq. [5] the stan- 
dard package of disordered statistical mechanics can 
be introduced: the partition function Zv(P;0 — 
Y^, a exp[— 0Hy(cr; £)], the Boltzmann state oj(.) = 
J2 a .e-^ Hv ^ /Z V (P;£), and the related free energy as 
A(/3,7,0) = limy^oo^E log (/?;£), where E aver- 
ages over the quenched distributions of the bit strings 
£. To obtain an explicit expression of the free energy, 
at first, trough the Hubbard-Stratonovick transform we 
map the partition function of our model into a bipar- 
tite ER one, whose parties are the V Ising spins and L 
auxiliary Gaussian fields z as 




a 



FIG. 3: (Color on line) Natural logarithm for the ratio c/c ER 
as a function of a and a for a system with V = 2000. When a 
follows the scaling of Eq.[4]with 7 = 1.5 (dashed line) the ratio 
is approximately 4, meaning large clustering. Data points (•), 
fitted by a power law ~ (l + a) v ,n w 1.5, demarcate the region 
where the graph is made up of 20 components (a macroscopic 
one plus isolated nodes). 
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then we enlarge the technique of the double stochastic 
stability |17j by introducing the following interpolating 
structure 



A(t) 



E 
V 



lQ gE / °°ri^(^)exp[t^?E ( 6 ) 

L V V L 

/ c — 1 z /b — 1 A* 

where now E = E^E, ; E X , bi c [with Z c <E (1, L)}, and 
[with lb £ (1, V)] are real numbers (possibly functions 
of /3, 7, 0) to be set a posteriori. 

Note that A(t = 1) is our goal, while A(t = 0) is straight- 
forward as it involves only one-body calculations. So we 
want to use the fundamental theorem of calculus to get 
a sum rule, as A(l) = A(0) + J^[dA(t')/df} t '=tdt, which 
ultimately implies the evaluation of the i-streaming of 
eq. (JsJ) . Now, we need to sort out our order parameters: 
as replica symmetry (RS) is expected to be conserved 
in ferromagnetic diluted networks, we (naturally) avoid 
(multi)-overlaps by defining M h = V^ 1 J2i ^h+i{^i) 
(and analogously for the other party trough Ni c ), where 
the index in u> means that we are considering all the pos- 
sible magnetizations built trough only lb + 1 links inside 
the graph (as the graph is no longer weighted, it is a mi- 
crocanonical decomposition of the observable in subclus- 
ters close to the one introduced in [IS])- Once introduced 
the averaged order parameters as (M) = P(lb)Mi b , 
and (N) = T,i c p (lc)Ni c , we find that d t A(t) = S(t) - 
VP~f/ 2 T,i b J2i c p (h)P(lc)M h Ni c ; where the fluctuation 
source S(t) is proportional to ((M — M)(N — N)) and 
can be neglected at the RS level (whose order parameter 
values are denoted via a bar, namely M,N). 
As a consequence it is possible to solve for the RS free 
energy to get 



A(/?, 7 ,0) = log 2 

h 2 



7 
2V e 



(logcosh(V/3W w )) 



(M)(N). 



Trough the self consistent relation (N) = ^2l(M), we 
can express the whole theory only via (M) (as expected 
looking at the original Hamiltonian ([5])) and we start 
discussing the various case of interest: 

• 8 = 0: Fully connected weighted regime 
The case 9 = reduces to the Curie- Weiss model 
and in particular, the upper bound on 7 (i.e. 7 = 2) 
describes the un- weighted fully connected topology. 
Choerently its termodynamics turns out to be 



2,6 

h 2 



= 0)=log2+|logcosh(^M) 
M 2 , M = tanh(^M). 



(7) 



Note that, as there is only one possible network 
built with all the links, all the subgraph magne- 
tizations collapse into only one, namely P(M) = 



8{M — Mew), where with Mew we mean the 
standard Curie- Weiss magnetization. Furthermore, 
note that for 7 = 2 we recover exactly the CW ther- 
modynamics. 

• 8 = 1/2: Standard dilution and ER regime 
With a scheme perfectly analogous to the previous 
one we can write down the free energy and its re- 
lated self-consistent equation as 



A{p, 7, = 1/2) = log 2 + lim ( -L= ■ (8) 

v=oo 2^V 

logcosh(^^<M)) - ^(M) 2 ), 

(M) = lim tanh(^v^(M)). (9) 

Ficticious diverging contributions emerge in the 
thermodynamic limit because the normalization we 
chose for the Hamiltonian ^ gives the correct ex- 
tensivity for the fully connected case only, while for 
9 = 1/2 it is 0{V- r ) on an ER graph. Note in fact 
that the argument of the logarithm of the hyper- 
bolic cosine can be read as flyJV (M) . As in stan- 
dard approaches |18j it is enough to renormalize 
the coupling by scaling :19j it with the amount of 
nearest neighbours (that in the ER dilution scales 
exactly linearly with the volume size). 

• 9 > 1/2: Extremely dilution regime and all the 
other cases of interest 

With a scheme analogous to the previous one it is 
possible to write down the pertaining free energy, 
coupled with its self-consistent relation for its ex- 
tremization. 

Interestingly, the effective field felt by the trial spin - 
obtained extremizing eq. Q with respect to (M)- scales 
as the square root of the coupling strength instead of a 
linear behavior (this effect disappears approaching the 
CW limit where vJ — > J). As the interaction matrix in 
the Hamiltonian has been normalized (i.e. is bounded by 
one) , V J > J, the local field is "higher with respect to a 
naive expectation" 21j. This feature, which is a conse- 
quence of the diverging variance in the bitstring distribu- 
tion [12] , however does not affect the transition line that 
scales consistently with a manifestation of a collective, 
global effect, as j3 c oc J^ 1 . 

In order to prove the last statement, we have to tackle the 
control of the fluctuations of the rescaled order parame- 
ters (M) = W(M - M), (AO = VZ(N - N): At first 
we need to derive the i-streaming of the squares of these 
objects (i.e. (M 2 ), (MAT), (A/ -2 )), which lead to a system 
of coupled ordinary differential equations in t, namely 

{M 2 ) = 2{M 2 ){MN) 1 (10) 
(MAO = (M 2 )(N 2 ) + ((MAT) 2 ), (11) 
(A/" 2 ) = 2(Af 2 )(MAf), (12) 
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whose Cauchy condition at t = can be obtained imme- 
diately, such that we can solve the system, evaluate it at 
t = 1 (the original statistical mechanics framework) and 
check where these fluctuations do diverge (identifying the 
expected second order phase transition). By using Wick 
theorem to express four point correlations trough series 
of two point ones and due to internal symmetries reflect- 
ing the mean field interactions among the two parties [12] 
the plan is fully solvable and all these fluctuations (and 



the correlation (A4AT) ) are found to diverge on the same 
line: /3 C = J -1 , as intuitively expected. 
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